---
title: "Lack of independence among multiple comparisons"
author: "A. Jonathan R. Godfrey"
date: "last updated: 16 September 2016"
bibliography: briefs.bib
output: html_document
---

```{r setup, purl=FALSE, include=FALSE}
library(knitr)
opts_chunk$set(dev=c("png"), fig.align="center")
opts_chunk$set(comment="", fig.path="./Figures/MultipleComparisons")
```


The easiest way to demonstrate the lack of independence among a set of multiple comparisons is to use three treatments, and compare two differences, as would Dunnett's procedure, or add the third difference as per Tukey's procedure.

Let us imagine an experiment was run 1000 times, and led to three different treatment means. Under the null hypothesis of equal treatment means, the difference between a pair of means would also have a mean of zero. Let us then also assume the treatment means are all random values taken from a standard normal distribution. (Adding a prescribed mean to each is possible.)
```{r}
T1=rnorm(1000)
T2=rnorm(1000)
T3=rnorm(1000)
```
We then calculate the differences
```{r}
D1=T1-T2
D2=T3-T2
```
The correlation of these differences is evident in the following figure. The differences have a correlation of `r round(cor(D1,D2),3)`. 
```{r DiffsPlotted, fig=TRUE, echo=FALSE, results="hide", fig.cap="Differences plotted against one another."}
plot(D2~D1)
```


For the record, we could find
```{r}
D3=T1-T3
```
and calculate the correlation with `D1` and `D2` to be `r round(cor(D1,D3),3)` and `r round(cor(D3,D2),3)` respectively. The signs of the correlations are dependent on the ordering of the treatment means in the differences, but they are fairly close to a magnitude of one half. To get closer to the theoretical value we could just force more runs in the above code.